Athens County
FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answering
Zhou, Wei, Mesgar, Mohsen, Adel, Heike, Friedrich, Annemarie
Table Question Answering (TQA) aims at composing an answer to a question based on tabular data. While prior research has shown that TQA models lack robustness, understanding the underlying cause and nature of this issue remains predominantly unclear, posing a significant obstacle to the development of robust TQA systems. In this paper, we formalize three major desiderata for a fine-grained evaluation of robustness of TQA systems. They should (i) answer questions regardless of alterations in table structure, (ii) base their responses on the content of relevant cells rather than on biases, and (iii) demonstrate robust numerical reasoning capabilities. To investigate these aspects, we create and publish a novel TQA evaluation benchmark in English. Our extensive experimental analysis reveals that none of the examined state-of-the-art TQA systems consistently excels in these three aspects. Our benchmark is a crucial instrument for monitoring the behavior of TQA systems and paves the way for the development of robust TQA systems. We release our benchmark publicly.
- Europe > United Kingdom > England (0.28)
- Asia > China (0.04)
- South America > Bolivia > Potosí Department > Tomás Frías Province > Potosí (0.04)
- (25 more...)
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (0.98)
- Information Technology > Artificial Intelligence > Natural Language > Question Answering (0.61)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.49)
Social Skill Training with Large Language Models
Yang, Diyi, Ziems, Caleb, Held, William, Shaikh, Omar, Bernstein, Michael S., Mitchell, John
People rely on social skills like conflict resolution to communicate effectively and to thrive in both work and personal life. However, practice environments for social skills are typically out of reach for most people. How can we make social skill training more available, accessible, and inviting? Drawing upon interdisciplinary research from communication and psychology, this perspective paper identifies social skill barriers to enter specialized fields. Then we present a solution that leverages large language models for social skill training via a generic framework. Our AI Partner, AI Mentor framework merges experiential learning with realistic practice and tailored feedback. This work ultimately calls for cross-disciplinary innovation to address the broader implications for workforce development and social equality.
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.14)
- North America > United States > California > Santa Clara County > Palo Alto (0.04)
- South America > Colombia (0.04)
- (9 more...)
- Health & Medicine > Therapeutic Area > Psychiatry/Psychology (0.68)
- Education > Curriculum (0.68)
- Education > Educational Technology (0.67)
- Education > Educational Setting > Higher Education (0.46)
The Interplay Between Symmetries and Impact Effects on Hybrid Mechanical Systems
Clark, William, Colombo, Leonardo, Bloch, Anthony
Hybrid systems are dynamical systems with continuous-time and discrete-time components in their dynamics. When hybrid systems are defined on a principal bundle we are able to define two classes of impacts for the discrete-time transition of the dynamics: interior impacts and exterior impacts. In this paper we define hybrid systems on principal bundles, study the underlying geometry on the switching surface where impacts occur and we find conditions for which both exterior and interior impacts are preserved by the mechanical connection induced in the principal bundle.
- North America > United States > Ohio > Athens County > Athens (0.04)
- North America > United States > New York (0.04)
- North America > United States > Michigan > Washtenaw County > Ann Arbor (0.04)
- (3 more...)
Strategic Client Selection to Address Non-IIDness in HAPS-enabled FL Networks
Farajzadeh, Amin, Yadav, Animesh, Yanikomeroglu, Halim
The deployment of federated learning (FL) within vertical heterogeneous networks, such as those enabled by high-altitude platform station (HAPS), offers the opportunity to engage a wide array of clients, each endowed with distinct communication and computational capabilities. This diversity not only enhances the training accuracy of FL models but also hastens their convergence. Yet, applying FL in these expansive networks presents notable challenges, particularly the significant non-IIDness in client data distributions. Such data heterogeneity often results in slower convergence rates and reduced effectiveness in model training performance. Our study introduces a client selection strategy tailored to address this issue, leveraging user network traffic behaviour. This strategy involves the prediction and classification of clients based on their network usage patterns while prioritizing user privacy. By strategically selecting clients whose data exhibit similar patterns for participation in FL training, our approach fosters a more uniform and representative data distribution across the network. Our simulations demonstrate that this targeted client selection methodology significantly reduces the training loss of FL models in HAPS networks, thereby effectively tackling a crucial challenge in implementing large-scale FL systems.
- South America > Brazil > Rio de Janeiro > Rio de Janeiro (0.04)
- North America > United States > Washington > King County > Seattle (0.04)
- North America > United States > Ohio > Athens County > Athens (0.04)
- (2 more...)
- Telecommunications (0.95)
- Information Technology > Security & Privacy (0.46)
Small jet engine reservoir computing digital twin
Wright, C. J., Biederman, N., Gyovai, B., Gauthier, D. J., Wilhelm, J. P.
Machine learning was applied to create a digital twin of a numerical simulation of a single-scroll jet engine. A similar model based on the insights gained from this numerical study was used to create a digital twin of a JetCat P100-RX jet engine using only experimental data. Engine data was collected from a custom sensor system measuring parameters such as thrust, exhaust gas temperature, shaft speed, weather conditions, etc. Data was gathered while the engine was placed under different test conditions by controlling shaft speed. The machine learning model was generated (trained) using a next-generation reservoir computer, a best-in-class machine learning algorithm for dynamical systems. Once the model was trained, it was used to predict behavior it had never seen with an accuracy of better than 1.8% when compared to the testing data.
- North America > United States > Ohio > Franklin County > Columbus (0.05)
- North America > United States > Ohio > Athens County > Athens (0.04)
- North America > United States > Ohio > Cuyahoga County > Cleveland (0.04)
- (6 more...)
Cumulative Reasoning with Large Language Models
Zhang, Yifan, Yang, Jingqin, Yuan, Yang, Yao, Andrew Chi-Chih
While language models are powerful and versatile, they often fail to address highly complex problems. This is because solving complex problems requires deliberate thinking, which has been only minimally guided during training. In this paper, we propose a new method called Cumulative Reasoning (CR), which employs language models in a cumulative and iterative manner to emulate human thought processes. By decomposing tasks into smaller components, CR streamlines the problem-solving process, rendering it both more manageable and effective. For logical inference tasks, CR consistently outperforms existing methods with an improvement up to 9.3%, and achieves an accuracy of 98.04% on the curated FOLIO wiki dataset. In the context of the Game of 24, CR achieves an accuracy of 98%, which signifies a substantial enhancement of 24% over the previous state-of-the-art method. Finally, on the MATH dataset, we establish new state-of-the-art results with 58.0% overall accuracy, surpassing the previous best approach by a margin of 4.2%, and achieving 43% relative improvement on the hardest level 5 problems (22.4% to 32.1%). Additionally, we expand the concept of Cumulative Reasoning to incorporate a Python code environment, deliberately omitting external aids such as retrieval and web browsing and focusing solely on the LLM's intrinsic reasoning capabilities within a Python code environment. Our experiments in this setting yielded impressive results, with an overall accuracy of 72.2% on the MATH dataset, significantly outperforming the PAL method with 38.8% relative improvement. Code is available at https://github.com/iiis-ai/cumulative-reasoning.
- North America > United States > New York > Bronx County > New York City (0.04)
- Europe > Western Europe (0.04)
- North America > United States > Ohio > Athens County > Athens (0.04)
- (9 more...)
- Media (1.00)
- Leisure & Entertainment (1.00)
- Health & Medicine > Therapeutic Area > Oncology (0.46)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Logic & Formal Reasoning (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Cognitive Science > Problem Solving (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.31)
An Expectation-Realization Model for Metaphor Detection
Uduehi, Oseremen O., Bunescu, Razvan C.
We propose a metaphor detection architecture that is structured around two main modules: an expectation component that estimates representations of literal word expectations given a context, and a realization component that computes representations of actual word meanings in context. The overall architecture is trained to learn expectation-realization (ER) patterns that characterize metaphorical uses of words. When evaluated on three metaphor datasets for within distribution, out of distribution, and novel metaphor generalization, the proposed method is shown to obtain results that are competitive or better than state-of-the art. Further increases in metaphor detection accuracy are obtained through ensembling of ER models.
- North America > United States > Illinois > Cook County > Chicago (0.05)
- Asia > Middle East > UAE > Abu Dhabi Emirate > Abu Dhabi (0.05)
- North America > United States > Ohio > Athens County > Athens (0.04)
- (10 more...)
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Delgado, Fernando, Yang, Stephen, Madaio, Michael, Yang, Qian
Despite the growing consensus that stakeholders affected by AI systems should participate in their design, enormous variation and implicit disagreements exist among current approaches. For researchers and practitioners who are interested in taking a participatory approach to AI design and development, it remains challenging to assess the extent to which any participatory approach grants substantive agency to stakeholders. This article thus aims to ground what we dub the "participatory turn" in AI design by synthesizing existing theoretical literature on participation and through empirical investigation and critique of its current practices. Specifically, we derive a conceptual framework through synthesis of literature across technology design, political theory, and the social sciences that researchers and practitioners can leverage to evaluate approaches to participation in AI design. Additionally, we articulate empirical findings concerning the current state of participatory practice in AI design based on an analysis of recently published research and semi-structured interviews with 12 AI researchers and practitioners. We use these empirical findings to understand the current state of participatory practice and subsequently provide guidance to better align participatory goals and methods in a way that accounts for practical constraints.
- North America > United States > New York > New York County > New York City (0.15)
- North America > United States > California > Los Angeles County > Los Angeles (0.14)
- North America > United States > Massachusetts > Suffolk County > Boston (0.05)
- (21 more...)
- Research Report > Experimental Study (1.00)
- Questionnaire & Opinion Survey (1.00)
- Instructional Material > Course Syllabus & Notes (0.67)
- Health & Medicine > Therapeutic Area (1.00)
- Education (1.00)
- Government (0.93)
- (3 more...)
GeoExplainer: A Visual Analytics Framework for Spatial Modeling Contextualization and Report Generation
Lei, Fan, Ma, Yuxin, Fotheringham, Stewart, Mack, Elizabeth, Li, Ziqi, Sachdeva, Mehak, Bardin, Sarah, Maciejewski, Ross
Geographic regression models of various descriptions are often applied to identify patterns and anomalies in the determinants of spatially distributed observations. These types of analyses focus on answering why questions about underlying spatial phenomena, e.g., why is crime higher in this locale, why do children in one school district outperform those in another, etc.? Answers to these questions require explanations of the model structure, the choice of parameters, and contextualization of the findings with respect to their geographic context. This is particularly true for local forms of regression models which are focused on the role of locational context in determining human behavior. In this paper, we present GeoExplainer, a visual analytics framework designed to support analysts in creating explanative documentation that summarizes and contextualizes their spatial analyses. As analysts create their spatial models, our framework flags potential issues with model parameter selections, utilizes template-based text generation to summarize model outputs, and links with external knowledge repositories to provide annotations that help to explain the model results. As analysts explore the model results, all visualizations and annotations can be captured in an interactive report generation widget. We demonstrate our framework using a case study modeling the determinants of voting in the 2016 US Presidential Election.
- North America > United States > California (0.14)
- North America > United States > Illinois > Cook County > Chicago (0.05)
- North America > United States > Ohio > Athens County (0.04)
- (6 more...)
- Questionnaire & Opinion Survey (0.93)
- Research Report > Experimental Study (0.68)
- Information Technology > Data Science > Data Mining (1.00)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Spatial Reasoning (1.00)
- Information Technology > Artificial Intelligence > Natural Language (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning > Regression (0.55)
New intelligent defense systems to reduce the risks of Selfish Mining and Double-Spending attacks using Learning Automata
Ghoreishi, Seyed Ardalan, Meybodi, Mohammad Reza
In this paper, we address the critical challenges of double-spending and selfish mining attacks in blockchain-based digital currencies. Double-spending is a problem where the same tender is spent multiple times during a digital currency transaction, while selfish mining is an intentional alteration of a blockchain to increase rewards to one miner or a group of miners. We introduce a new attack that combines both these attacks and propose a machine learning-based solution to mitigate the risks associated with them. Specifically, we use the learning automaton, a powerful online learning method, to develop two models, namely the SDTLA and WVBM, which can effectively defend against selfish mining attacks. Our experimental results show that the SDTLA method increases the profitability threshold of selfish mining up to 47$\%$, while the WVBM method performs even better and is very close to the ideal situation where each miner's revenue is proportional to their shared hash processing power. Additionally, we demonstrate that both methods can effectively reduce the risks of double-spending by tuning the $Z$ Parameter. Our findings highlight the potential of SDTLA and WVBM as promising solutions for enhancing the security and efficiency of blockchain networks.
- North America > United States > California > San Francisco County > San Francisco (0.14)
- Asia > Middle East > Iran > Tehran Province > Tehran (0.04)
- North America > United States > Oklahoma > Cleveland County > Norman (0.04)
- (5 more...)
- Information Technology > Security & Privacy (1.00)
- Government > Military (0.64)
- Education > Educational Setting > Online (0.34)
- Information Technology > e-Commerce > Financial Technology (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Reinforcement Learning (0.50)
- Information Technology > Artificial Intelligence > Machine Learning > Computational Learning Theory > Learning Automata (0.42)
- Information Technology > Enterprise Applications > Human Resources > Learning Management (0.34)